Information Relaxations and Dynamic Zero-Sum Games

Dynamic zero-sum games are an important class of problems with applications ranging from evasion-pursuit and heads-up poker to certain adversarial versions of control problems such as multi-armed bandit and multiclass queuing problems. These games are generally very difficult to solve even when one player’s strategy is fixed, and so constructing and evaluating good sub-optimal policies for each player is an important practical problem. In this paper, we propose the use of information relaxations to construct dual lower and upper bounds on the optimal value of the game. We note that the information relaxation approach, which has been developed and applied successfully to many large-scale dynamic programming problems, applies immediately to zero-sum game problems. We provide some simple numerical examples and identify interesting issues and complications that arise in the context of zero-sum games.